AD  PO 008  47 


Combining  Results  of  Independent  Research  in  Tank 
Crewman  Performance 


Barbara  A.  Black 
U.  S.  Army  Research  Institute 


Charlotte  K.  Campbell 
Human  Resources  Research  Organization 


le  effectiveness  of  any  combat  weapon  system  is  in  large  measure  a 
function  of  the  level  of  performance  of  the  soldier  operator.  The  US  Army  is 
interested  in  ensuring  maximum  effectiveness  of  the  new  Ml  tank  system  by 
optimally  selecting  and  training  Ml  crewmen.  To  support  the  Army's  effort  in 
maximizing  system  effectiveness  from  the  personnel  selection  aspect,  the  Army 
Research  Institute  has  conducted  extensive  research  in  the  area  of  tank  crew- 
man performance  prediction  during  the  past  several  years.  The  purpose  of 
this  paper  is  to  evaluate  the  results  of  this  research  in  order  to  determine 
by  job  content  area,  for  both  trainees  and  job  incumbents,  whether  quantifi- 
able aptitudes  are  related  to  tank  gunnery  performance. 


Black  and  Kraemer  (1981)  identified  three  aptitude  categories  which 
potentially  underlie  gunnery  performance.  These  included  a cognitive  compo- 
nent as  encountered  in  troubleshooting,  a perceptual  component  as  in  target 
acquisition  and  a psychomotor/perceptual-motor  component  as  in  target  track- 
ing. Each  of  the  four  crew  positions  within  the  tank  system  (i.e.,  loader, 
driver,  gunner  and  tank  commander)  requires  performance  of  tasks  which  appear 
to  contain  these  components,  albeit  in  varying  degrees.  A review  of  the 
Armor  crewman  performance  prediction  literature  lends  support  to  this  cate- 
gorization but  points  to  an  additional  dichotomy  with  reference  to  research 
techniques  utilized.  Techniques  include  paper-and-pencil  tests  as  well  as 
tests  called  job  samples  which  require  either  simulators  or  actual  tank 
equipment . 


These  aptitude  categories  and  research  techniques  were  identified  in  the 
tank  crewman  performance  prediction  literature.  In  the  area  of  cognitive 
testing,  the  literature  included  validation  of  ASVAB-derived  composite  scores 
such  as  CO,  GT  and  AFQT  as  paper-and-pencil  predictors  of  gunnery  performance 
(Greenstein  & Hughes,  1977;  Campbell  & Black,  1982;  Black,  in  preparation), 
and  simulator  based  tests  of  the  tank  fire  control  computer  (Campbell  & 
Black,  1982;  Black,  in  preparation).  For  perceptual  testing,  paper-and- 
pencil  tests  are  also  the  most  commonly  encountered  (Greenstein  & Hughes, 
1977;  Eaton,  1978;  Eaton,  Bessemer,  & Kristiansen,  1979),  although  two 
instances  of  simulator  based  perceptual  tests  were  found  (Eaton,  Johnson,  & 
Black,  1980;  Campbell  & Black,  1982) . Validation  of  psychomotor  tests  using 
hands-on  equipment  can  be  found  in  three  reports  (Eaton,  1978;  Kress,  1980; 
Black,  in  preparation),  and  finally,  simulation  techniques  are  applied  to 
psychomotor  performance  prediction  in  two  reports  (Eaton  et  al.,  1980; 
Campbell  & Black,  1982;  Black,  in  preparation).  The  correlations  reported 
for  these  research  efforts  provided  the  data  for  the  meta-analyses. 

Method 


The  eight  documents  included  in  the  review  of  Armor  crewman  performance 
prediction  literature  produced  a total  of  18  data  sets  for  evaluation.  Data 
sets  were  accepted  for  meta-analysis  based  upon  the  following  criteria: 
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1)  predictor  variables  were  obtained  from  tests  which  could  be  classified  as 
either  cognitive,  perceptual  or  psychomotor/perceptual-motor,  2)  criterion 
measures  were  tank  live  fire  gunnery  hit  scores,  and  3)  subjects  were  either 
tank  gunner  trainees  or  operational  unit  gunner/TCs. 

Data  sets  were  placed  into  analytic  categories  according  to  the  format 
presented  in  Table  1.  Each  data  set  had  between  one  and  ten  correlations 
that  were  used  in  the  meta-analyses  for  each  analytic  category. 

Table  ’ 

Number  of  Correlations  (and  Data  Sets) 

Available  for  Meta-Analysis 


APTITUDE 

TEST  TYPE 

CATEGORIES 

P ap  er -and-P enc il 

Job  Sample 

Cognitive 

18  (11) 

8 (2) 

Perceptual 

63  (10) 

6 (6) 

Psychomotor  or 
Perceptual-Motor 

- 

41  (11) 

Two  methods  were  used  for  combining  and  evaluating  the  results  reported 
in  the  literature.  The  first,  drawn  from  Rosenthal  (1978),  used  exact 
probabilities  (one-tailed)  of  the  correlations  to  compute  an  overall  Z_  for 
each  data  set;  the  exact  probabilities  were  corrected  for  the  number  of 
correlations  drawn  from  each  data  set  in  each  analytic  category.  The 
Z^-values  for  each  data  set  in  each  category  were  then  combined  using  a method 
whereby  each  is  weighted  by  the  degrees  of  freedom  of  its  respective  data 
set.  The  method  yields  a Z_  for  each  analytic  category  (see  Table  2)  . 

The  second  method  was  based  on  Glass  (1977),  who  advocates  the  averaging 
of  correlation  coefficients  or  coefficients  of  determination.  Here,  the 
Fisher  ^-scores  were  computed  for  each  correlation  and  combined  (Snedecor  & 
Cochran,  1967)  first  within  data  sets  and  then  across  data  sets  within  each 
category  to  yield  an  overall  weighted  average  z_.  This  value  was  then 
converted  back  to  a correlation;  the  squared  correlations,  representing  the 
proportion  of  variance  accounted  for,  are  reported  in  Table  2. 

Results  and  Discussion 


While  the  aggregated  results  of  cognitive  paper-and-pencil  testing 
produced  a statistically  significant  cumulative  Z for  trainees,  it  is 
interesting  to  note  that  the  average  variance  in  gunnery  scores  accounted  for 
by  the  cognitive  component  is  only  2.5%.  So  although  the  predictions  are 
consistent  and  reliable,  they  do  not  provide  very  much  information.  One 
variable  of  the  cognitive  job  sample  tests,  computer  accuracy,  was  a signifi- 
cant predictor  for  operational  unit  personnel,  accounting  for  over  10%  of  the 
variance  in  gunnery  performance,  but  the  variable  is  not  a significant 
predictor  for  trainees. 
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Table  2 

Results  of  Two  Meta-Analysis  Techniques  Relating 
Tank  Crewman  Aptitudes  to  Tank  Gunnery  Performance 


OPERATIONAL 

UNIT  SOLDIERS 

TRAINEES 

Variance 

Z Accounted  for 

Z 

Variance 
Accounted  for 

COGNITIVE 

Paper  & Pencil  Tests 

1.511 

2.5% 

2.171* 

2.5% 

Job  Sample  Tests 

.Ml  Computer  Accuracy 

2.106* 

10.6% 

.977 

0.4% 

.Ml  Computer  Speed 

-.627 

2.5% 

1.079 

0.8% 

PERCEPTUAL 

Paper  & Pencil  Tests 

-6.741 

0.0% 

-4.957 

0.1% 

Job  Sample  Tests 

•Round  Sensing 

- 

— 

3.002*** 

4.8% 

PSYCHOMOTOR/PERCEPTUAL-MOTOR 
Job  Sample  Tests 

.Tracking  Accuracy 

1.441 

6.0% 

1.266 

1.0% 

.Tracking  Speed 

-.122 

0.0% 

1.388 

0.9% 

.Main  Gun  Lay  Accuracy 

2.542** 

7.1% 

- 

- 

•Main  Gun  Lay  Speed 

2.239* 

4.9% 

- 

- 

.Target  Engag.  Hits 

-.433 

0.1% 

.581 

0.1% 

.Target  Engag.  Speed 

- 

- 

.245 

0.0% 

.Sub-Caliber  Hits 

-.239 

0.1% 

- 

- 

.Sub-Caliber  Speed 

1.547 

4.0% 

— 

“ 

**jj  < .01  one-tailed 
***j>  < .001  one-tailed 


Perceptual  paper-and-pencil  tests  were  poor  predictors  of  gunnery  scores 
for  both  operational  unit  personnel  and  trainees.  The  job  sample  test 
approach,  however,  produced  positive  correlations  in  all  data  sets,  for  a 
highly  significant  effect,  but  the  variance  accounted  for  averages  less  than 
5%.  Whether  the  approach  would  be  effective  among  operational  unit  personnel 
is  unknown.  Two  of  the  job  sample  tests  of  psychomotor/perceptual-motor 
aptitude  were  significant  predictors  across  studies  for  operational  unit 
soldiers,  but  none  was  a predictor  for  trainees. 

Overall,  it  would  appear  that  job  sample  tests  are  better  predictors  of 
performance  by  job  incumbents  than  are  paper-and-pencil  techniques.  Tor 
trainees,  however,  where  performance  is  usually  measured  during  their  ear- 
liest experience  on  the  tank,  hands-on  tests  are  sometimes  predictive,  and  so 
are  paper-and-pencil  tests.  It  should  b«.  noted  that  no  attempt  was  made  to 
separate  concurrent  predictions  and  actual  time-separated  predictions  for  the 
analyses  of  unit  personnel  performance.  And  because  perceptual  paper-and- 
pencil  tests  were  combined  within  data  sets  and  adjusted  for  that  process  of 
combining,  the  large  numbers  of  small  correlations  in  each  data  set  caused 
the  combined  Zs  for  the  sets  to  be  very  large  negative  numbers.  Examination 
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of  individual  tests  across  research  efforts  could  lead  to  different  conclu- 
sions for  a few.  In  general,  meta-analysis  techniques  appear  to  be  valuable 
tools  in  assimilating  independent  research  results  and  providing  insight  for 
future  research  efforts. 
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